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ABSTRACT 

The use of technology such as laptop computers in the classroom has long been recognized as 
destructive behavior since it diverts a student’s attention from course topics. However, it is 
conceivable that every student will be using some form of technology in the near future. 
Determining the effects of interactive software on students’ learning outcomes can have a 
profound effect on engineering education. The ultimate aim of this research is to transform 
students into active learners who are able to better comprehend, are less distracted, and can 
achieve higher academic performance. In this study, first year engineering students used online 
metacognition software while interactively participating in the classroom. Both qualitative and 
quantitative methods using the pre- and post-test experimental designs as well as a debriefing 
questionnaire were utilized. The academic achievement of students’ through the integration of 
interactive technology was the output variable, while the input variables were divided into four 
categories: students’ understanding of the concepts, confidence level, apprehension level, and 
motivation. In addition, this study also examined the amount of class participation to measure 
students ’ communication apprehension and its correlation to academic performance. 

In order to improve students' learning outcomes using metacognitive strategies, it was discovered 
that the use of interactive technology followed by group discussions and class assignments greatly 
enhanced students' comprehension of scientific facts and their ability to explain them. In addition, 
the current study showed that engineering students' communication apprehension was also 
reduced resulting in improvement in confidence and motivation towards academic success. 
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INTRODUCTION 



he recent increase of laptops and other similar devices in the classroom has virtually replaced manual 
note taking. Technology in the classroom has been demonstrated to enhance teacher and student 
technology literacy, student responsibility, and student independence. The quality of a student’s 
work and the content of his/her presentations have improved as well (Fouts & Stuen, 1997; Gardner, Morrison, 
Jarman, Reilly & McNally, 1994). As laptops gain popularity in the classroom, so too is the use of multimedia and 
presentation software since it offers students and educators a unique way to introduce complex subject matters to the 
novice. Multimedia software using graphics, videos, and animations provides the ability to interact electronically 
with students with different learning styles (Ayersman, 1996; Provenso, Brett & McCloskey, 1999). This 
constructivist approach encourages the learning of the material content through more sophisticated cognitive 
interactions (Siegle, Foster, 2001). Constructivism involves learning material based upon the experiences of those 
learning to construct much of what they understand fSchunk, 2000). 


The use of interactive software such as LectureTools can sculpt students into active learners (Thorsen, 
1998) by empowering them with the ability to direct their learning through the use of sophisticated cognitive 
strategies such as metacognition. There are two basic elements to metacognition: knowledge of cognition and 
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regulation of cognition (Flavell, 1976). Knowledge of cognition refers to “what” individuals know about their own 
cognition. Furthermore, knowledge of cognition is composed of three types of awareness: (1) declarative: knowing 
what components dominate one’s own performance; (2) procedural: knowing how to do things; and (3) conditional: 
knowing the why and when aspects of cognition (Brown 1987). Knowing when and what to rehearse is a strategy 
employed by successful students. The second basic element of metacognition involves regulation of cognition, 
which involves controlling a set of activities that facilitates and improves one's learning. Metacognitive regulation 
enhances achievement in several ways. First, students’ display better handling of their attentional resources; second, 
students demonstrate greater execution of existing strategies; third, students display an enhanced understanding of 
comprehension breakdowns (Schraw, 1998). A number of studies have concluded that improvement of one's 
regulatory skills significantly improves one’s learning. Moreover, it was reported that younger students are better 
able to acquire metacognitive skills if such skills are taught as part of classroom instruction (Cross 1988, Brown, 
1989). 

LITERATURE REVIEW 

The use of multimedia based instruction in biology, engineering, anatomy, and physiology classrooms 
resulted in a higher level of understanding of the subject matter and improved exam scores. Ritt and Stewart (1996) 
observed that students who used technology in the classroom outperformed their counterparts by scoring, on 
average, 10 points higher on their exams. They also reported 9 out of 10 students who used different multimedia 
learning software improved their understanding of the course materials. Lunsford (1979) observed that basic writers 
function at a cognitive developmental level below what is required to achieve abstract thinking. Students who have 
not mastered the basic learning skills are unable to successfully apply the necessary principles required to succeed at 
the college level. This effects their motivation to learn, most likely converting it from intrinsic to extrinsic 
motivation (Logan et ah, 2010). Intrinsic motivation is defined as the internal motivation within an individual such 
as curiosity, whereas extrinsic motivation is defined as the external motivating factors such as grades (Logan et ah, 
2010). Wittrock (1986) observed that in order to develop those schemata, students need to learn and consciously 
employ strategies or sequences of thought that encourage them to engage with different types of written text. Lepper 
(2005) discovered that intrinsic motivation was positively correlated to students’ academic performance and that 
extrinsic motivation was negatively correlated to academic performance. Lau and Chan (2003) demonstrated that 
good and poor readers differ significantly in their levels of intrinsic reading motivation but not extrinsic motivation. 
As Logan (2010) observed, a student's motivation plays a greater role in his/her performance and the development of 
reading comprehension skills. 

In Musical Excellence . Aaron Williamson (2004) discussed how every good musician needs to have the 
necessary skills to concentrate, set goals, set up flexible practice schedules, and most of all, have the ability to self- 
evaluate. These attributes are dependent upon the motivation of individual students, which has also been observed as 
an important factor in overall academic performance during this study. 

METACOGNITION 

Metacognition strategies are management techniques by which learners control their learning processes 
through planning, monitoring, and evaluating approaches (Rubin 1990). Planning includes choosing suitable 
strategies such as paying careful attention before starting a task. Monitoring includes one’s understanding of 
comprehension and task accomplishment such as self-testing during learning. Evaluating refers to assessing the 
knowledge acquired and proficiencies of one’s learning (Schraw, 1998). These strategies center on focusing, 
planning, and evaluating one's own progress as measured by academic performance (Flavell 1976). The difference 
between cognitive and metacognitive knowledge (Flavell 1979) is that the metacognitive approach focuses on how 
the knowledge is used by students. Metacognition refers to knowing what you know or thinking about thinking. 
Cognitive strategies assist an individual in attaining a certain goal, such as comprehending a text, while 
metacognitive strategies assure achievement of the goal by quizzing oneself to assess understanding for example 
(Livingston 1997). Cognitive approaches suggest that low self-perceived ability inhibits achievement and high self- 
perceived ability fosters and encourages higher levels of learning and attainment of goals. 
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As students enter college, many of them refrain from asking questions to improve their level of 
understanding in the materials being presented in the classroom. Georgios Sideridis (2006) demonstrated in his 
study of 538 students that some students are overwrought with the “ought-self’. He describes the "ought-self' as 
being “obliged to engage in an activity that is grounded on fear and is associated with a network of avoidance- 
related behaviors.” This explains the significant amounts of variability in task avoidance, performance avoidance, 
and fear of failure. Students demonstrating strong “ought-self’ behavior were negatively associated with 
achievement and positively associated with indices of anxiety and depression. Students who are intrinsically 
motivated tend to employ more metacognitive strategies than students who are extrinsically motivated. Other studies 
have found that students who are extrinsically motivated through rewards eventually lose intrinsic interest (Grolnick 
& Ryan, 1987). Some of these intrinsically motivated values “come from an individual’s formation of a cognitive 
schema that is integrated into one’s values which lead to fully self-determined behaviors” (Sideridis, 2006). 

RESEARCH GOAL 

The aim of this study was to engage students’ metacognitive strategies in order to improve learning and 
academic performance. The study also examined the communication apprehension levels of students to determine if 
they inhibited class participation. It also attempted to understand if the causes of underachievement are due to 
communication apprehension, motivational deficits, or failure to use metacognitive strategies. 

DESIGN OF THE STUDY 

Methodology 

This study used both qualitative and quantitative methods using the pre- and post-test experimental designs 
as well as a debriefing questionnaire. Academic achievement through the integration of interactive technology was 
the dependent variable. Students’ understanding of the concepts, confidence level, apprehension level, and 
motivation to learn were the four independent variables used. 

1. Students’ understanding of engineering concepts and phenomenon, which was measured in two levels- 
sUidents either grasp the material or they did not. 

2. Students’ confidence level (which was measured by the number of hours spent studying for exams and 
preparing for class), class participation (which was measured by the number of questions asked), and the 
grade they expected on an exam (which was examined before lecture using pre-test questionnaires and after 
lecture using post-lecture questionnaires). 

Two other areas that were examined in this study involve: 

1. Students’ apprehension level - analyzed by converting confidence level scores and comparing them to 
exam scores to determine level of apprehension and possibly communication apprehension. 

2. Students’ motivation to learn engineering - examined by questions 1,3,5,7. 

Identical pre and post- test questionnaires were used to evaluate comprehension, confidence levels, and 
motivation to learn the materials presented in the classroom. The questionnaire (Appendix A) examined students' 
understanding of metacognition concepts as well as their confidence levels. The second part, the debriefing 
questionnaire (Appendix B) examined students’ motivation to use technology in the classroom. There were nine 
Likert type questions, one multiple choice question, one yes/no question and one open ended question used to 
determine the students’ level of intrinsic motivation: 

a. Personal interest and enjoyment of the content (questions 1, 2, 12*) 

b. Motivation to use technology (questions 3, 4, 5, 6, 7*, 9*, 11*) 

c. Ability to effectively communicate (question 8) 
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The questions marked with an asterisk were negative statements and have been recorded for statistical 

analysis. 


Prior to taking their mid-term exam, participants were asked to complete a pre-test survey that was 
designed to measure their level of ability and confidence in the subject matter. Following the exam, the participants 
were asked to complete a post-test survey which was identical to the pre-test questionnaire and designed to assess 
the level of accuracy of their knowledge. The results of the pre-test were compared to mid-term exam scores. Later 
the mid-term exam scores were compared to the post-test survey to determine how accurately students recorded their 
assessment of understanding the subject matter. 

As part of the study, both the pre and post-test questionnaires were administered before and after each 
lecture using interactive software. Also, this occurred before and after each quiz and exam. 

RESEARCH METHODS 

This research was conducted in two phases: 1. a pilot study to establish the research tools’ reliability and 
validity, and 2. the main study to answer the research questions. Both phases used similar lecture materials presented 
by the same professor. The pilot study had 23 students registered for the class (21 males and 2 females); the main 
study consisted of 32 students (28 males and 4 females). The metacognition based software Lecture Tools 
( http://www.lecturetools,com ) provided the interactive environment for the students. The interactive lecture was 
presented once a week for each topic taught in class. Each interactive session started with a pre-test and was 
followed by a post-test as shown in Figure 1. At the end of the class students met in groups of three or four to 
discuss the lecture materials and to answer the post-test 



Figure 1: Overview Of Research Method 
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Using the metacognition based software, students were able to view the lecture on their laptop and provide 
real-time feedback about their level of understanding of the topics presented. Based on the students’ feedback, the 
professor reviewed topics as necessary in order to enhance the learning process. Students were also able to ask 
questions in an interactive mode while remaining totally anonymous. The results showed a significant increase in the 
number of students who asked questions using the online metacognition tool. 


Table 1: Comparison Of Exam Scores, Hours Spent Studying, 


And Confidence Levels Of Students With And Without Technology 



Technology vs. No Technology 

N 

Mean 

Std. Deviation 

Std. Error Mean 

% Score on Exam 

No Technology 

23 

78.622 

12.8348 

2.6762 

Technology 

32 

84.844 

16.5458 

2.9249 

Hours studied 

No Technology 

17 

2.88 

1.317 

.319 

Technology 

32 

3.02 

2.172 

.384 

Expected score 

No Technology 

12 

60.00 

28.920 

8.348 

Technology 

32 

33.94 

6.618 

1.170 


Table 1 shows that students who did not receive instruction through the use of technology scored lower 
than students receiving instructions through the use of technology. However, students who did not receive 
technology expected a higher grade than those who did receive technology. Although these students did not score 
higher on their exams, they had a higher level of confidence in their self- assessment. 

RESEARCH POPULATION 

The research population consisted of fifty-five students. Phase one consisted of first year engineering 
students signed up for EGR102 (N = 23) and phase two of the study also consisted of freshman engineering students 
signed up for the same class (N = 32). Two groups of freshman engineering students were studied over the course of 
two years. They alternately employed the use of laptop computers in the classroom with multimedia software to 
enhance their comprehension of engineering concepts taught at the University of Michigan-Flint. Each group used 
their laptop computers for the first half of the semester but switched to the traditional methods of note taking during 
the second half of the semester. The course grades of the two groups were compared at two weeks intervals during 
each semester and at the end of each semester. 

RESULTS 

To test the hypothesis that low perceived ability impedes performance, statistical analyses were performed 
based on intra-individual correlation coefficients between pre-test, exam and post-test performances of each 
participant. The analysis results indicated that students benefited from using interactive software and reviewing 
course material in groups after it was presented. 

The following section is categorized into four subsections to answer each of the research questions. The 
first subsection describes students’ understanding of engineering concepts by measuring the accuracy of their self- 
assessment of their level understanding of the subject matter in the post-test questionnaire. The second subsection 
measures the students’ confidence levels of the subject before and after each lecture and exam. The third subsection 
examines students’ communication apprehension by comparing class participation to final score in the class. The 
fourth subsection analyzes the students’ intrinsic motivation to learn engineering as retrieved from the questionnaire. 

Students’ Understanding Of Engineering Concepts And Phenomenon 

The statistical method Cronbach’s Alpha was used to estimate the internal consistency of the five-items 
used to determine the confidence level of students. The results showed the value of alpha to be 0.644, indicating a 
medium degree of internal consistency. The means of the individual items ranged from 1.54 to 3.04, with a mean on 
the total scale of 12.36 (SD = 3.76). Overall, the participants’ responses on the scale indicated that they possess a 
mid-range level of confidence with regard to their understanding of the subject matter (Mazumder & Ainsworth, 
2010 ). 
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One of the hypotheses was that students might regulate their own learning by improving their ability to 
self-assess their understanding. A two-way ANOVA was also performed to evaluate whether the amount of time 
spent studying for exams and confidence levels had an impact on academic performance on exams. The analysis 
results revealed a significant main effect of hours spent preparing for an exam and the grade expected, F (1, 2) = 
8.00, p<. 05. In other words, students who spent more time preparing expected to receive higher grades. 


Estimated Marginal Means of GradeExpected 


100 . 00 ' 



less than 1 hour 1 -3 hours 

Hourspreparingtest 


4-8 hours 


ActualMidTermGrade 

-58.67 

61.36 
61.39 

- 61.43 

65.09 

- 71.58 

73.49 
80.60 
81.90 

- 90.02 

90.10 


Non-estimable means are not plotted 

Figure 2: Means Of Grade Expected On Exam Vs. Hours Spent Studying For Exam 


Table 2: Correlations Of Total Questions Asked To Final Exam Scores 



Final Exam 

Total Quest Asked 

Spearman's rho 

Final Exam 

Correlation Coefficient 

1.000 

.437“ 

Sig. (2-tailed) 


.012 

N 

32 

32 

Total Quest Asked 

Correlation Coefficient 

.437“ 

1.000 

Sig. (2-tailed) 

.012 


N 

32 

32 


*. Correlation is significant at the 0.05 level (2-tailed). 


Due to non-normal distributions of grade and questions asked, non-parametric tests were conducted. The 
correlation between the frequency of total questions asked and the final exam score is presented in Table 2. The 
frequency of questions asked in person and the frequency of questions asked online were also compared to exam 
grades. The results indicate a positive correlation between the number of questions asked by students and their final 
exam score. In other words, students who asked more questions also did better on the final exam. This is statistically 
significant as the correlation coefficient is r (30) = +1.0, p<. 05. The coefficient of determination (r 2 ) was calculated 
to determine the proportion of variance in one variable that is accounted for by another variable (Jackson, 2005). 
The coefficient of determination of r 2 = 19.0% shows 19% of the variation in the final exam score is explained by 
the total number of questions asked in class, p=.01. 
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Table 3: Relationship Between Number Of Questions Asked Online To Final Exam Scores 



Final Exam Score 

Questions Asked online 

Spearman's rho 

Final Exam Score 

Correlation Coefficient 

1.000 

.516" 

Sig. (2-tailed) 


.002 

N 

32 

32 

Questions Asked Online 

Correlation Coefficient 

.516" 

1.000 

Sig. (2-tailed) 

.002 


N 

32 

32 


**. Correlation is significant at the 0.01 level (2-tailed). 


The correlation between number of questions asked online to final exam score is presented in Table 3 to 
understand how the metacognition based Lecture Tools assisted students in participation and improvement of their 
academic performance. The total number of questions asked online are related to final exam scores by a +1.0 which 
suggests a strong positive correlation between the total number of questions asked online in relation to final exam 
scores. 


A Spearman correlation coefficient was calculated for the relationship between the number of questions 
asked online and the final exam scores. This is statistically significant at r (30) = .516, p<. 01, indicating a 
significant linear relationship between the two variables. 



Figure 3: Correlation Between The Questions Asked Online And Final Exam Scores 


The positive correlation of +1 indicates that as the number of questions online increases, final exam scores 
also increase. The coefficient of determination was calculated as r 2 = .266, which indicates that 26.6% of the 
variation in final exam score, is explained by the total number of questions asked online (Jackson, 2005). Although, 
it did not appear that the numbers of questions asked online and the numbers of questions asked by hand are related. 
However, due to the fact that correlation coefficients only represent linear relationships, we are unable to conclude 
that there is no relationship between the two variables as Figure 4 shows that the variables have a curvilinear 
relationship. 
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The concave line above the fit line (middle line) represents the questions asked online while the curved line 
below the fit line represents the total number of questions asked by hand. To understand the relationship additional 
tests should be conducted to determine the significance of this relationship. 

Students’ Communication Apprehension 

This subsection of the study examined students’ apprehension level. Apprehension level was analyzed by 
comparing students' confidence level scores with their exam scores. It was observed during the study that students 
who displayed communication apprehension may gradually withdraw from the learning process by not engaging in 
the classroom. Students who participated in class by asking questions and becoming involved with the interactive 
software seemed to retain a better overall comprehension, hence resulting in higher academic achievement than 
those who chose to not actively participate. 


Table 4: Correlation Of Apprehension Level Vs. Final Class Score 



Total Anxiety X? 

Final actual score X? 

Spearman's rho 

Apprehension Level 

Correlation Coefficient 

1.000 

-.681” 

Sig. (2-tailed) 


.000 

N 

32 

32 

Final actual score 

Correlation Coefficient 

-.681*' 

1.000 

Sig. (2-tailed) 

.000 


N 

32 

33 


**. Correlation is significant at the 0.01 level (2-tailed). 
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The relationship between students’ apprehension level and final exam score is presented in Table 4. The 
analysis results showed r (30) = -.681, p<.01 indicating a significant linear relationship between the two variables. A 
strong negative correlation between apprehension level and final class grade explains why students with higher 
levels of apprehension had lower scores on the exam. Students who scored higher on the final exam had lower 
apprehension levels as shown in Figure 5. 



Figure 5: Correlation Of Students’ Apprehension Levels And Final Grade 


The large magnitude of Figure 5, shows a strong negative correlation between overall apprehension levels 
and final grade. The coefficient of determination, r 2 = .464 indicates that 46.4% of the variation in final class score 
is explained by the students’ apprehension level. 

Students’ Intrinsic Motivation To Learn Engineering 

The response to the debriefing (Appendix B) reflects students’ motivation to learn a new subject through 
intrinsic motivation as well as their willingness to experience the interactive software through the use of a laptop in 
the classroom. 

Figure 6 shows several students had mixed responses to the questions. The students expecting a grade in 
the eighty percentile consistently answered five or above on all of the motivation questions. A score of five or above 
indicated that they are in strong agreement with the statement made in the questionnaire. 
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Figure 6: Students Motivation To Learn Through Intrinsic Motivation 


SUMMARY AND CONCLUSION 

The aim of this study was to engage students’ metacognitive strategies to improve student learning. In this 
study, it was discovered that the use of interactive technology followed by group discussions and class assignments 
greatly enhanced students’ comprehension of scientific facts and their ability to explain them. In addition, the study 
concluded that students who studied engineering through the use of interactive software greatly reduced their 
communication apprehension. This resulted in a building of confidence and development of greater motivation to 
learn the subject in relation to self-motivation, which is important to the student’s future career goals. 

Although it is challenging to assess whether the positive results are due to the use of laptops and interactive 
software in the classroom, it is obvious that the interaction positively contributed to the students paying more 
attention. The increased interactions improved their level of understanding of the subject matter, which resulted in 
higher exam scores. On the other hand, it is possible that clarification and further understanding of any questions 
could have resulted from the group discussion and not necessarily from the use of technology. 

Even though it is well-known that top performing students already employ these strategies, the use of 
technology to trigger higher level of cognitive thinking may not be useful for that task if most of the students are 
already in the habit of planning, monitoring, and evaluating their learning. However, it may be useful for future 
studies to determine if a student’s motivation may conceivably play a greater role in their performance even if they 
have lower abilities. Since few studies have been conducted in this area, this would be an excellent topic to conduct 
future studies. In addition, more studies need to be conducted in the area of communication apprehension and 
developing methods of teaching students how to engage and apply a higher level of cognitive thinking to their 
education. 
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• Which of the following is not part of metacognition? 

a) Thinking about thinking 

b) Examining your brains processing 

c) Becoming a more strategic thinker 

d) Rote memorization 

• There are 3 stages of metacognition, which of the following is not one of these stages? 

a) Planning 

b) Monitoring 

c) Evaluating 

d) Learning 

• The self-regulation process is 

a) When you are not aware of your strengths and weaknesses 

b) Students assessing their own abilities 

c) Not helpful in informing you what you need to spend more or less time on 

• Metacognition 

a) Stresses active participation in problem solving 

b) Increases knowledge about learning and problem solving 

c) Increases capability of students to assess their own level of understanding 

d) All of the above 

APPENDIX B: Lecture Tools: Debriefing 

1. ) Which of these resources did you utilize outside of class throughout the semester? (check all that apply) 

a. ) Podcasts 

b. ) Online text books 

c. ) Other online resources 

d. ) Other hardcopy textbooks 

e. ) Lecture Tools slides from class, unaltered 

f. ) Lecture Tools slides from class, with additional notes 

g. ) Your hand-written /typed noted from class 

h. ) The hand-written/typed notes from class of classmates 

i. ) Studying one-on-one with the same classmate 

j. ) Studying one-on-one with multiple classmates 

k. ) Studying with more than one classmate as a small group 

l. ) Speaking with students who had previously taken the course 

m. ) Office hours or scheduled meetings with the instructor or teaching assistants 

n. ) Other, please specify 

2. ) On average, how much time each week did you spend studying for this course outside of class? 

a. ) Less than 1 hour 

b. ) 1-2 hours 

c. ) 3-5 hours 

d. ) 6-8 hours 

e. ) 9 or more hours 

3. ) How often did you use a laptop in this class 

a. ) Every lecture 

b. ) Most lectures 

c. ) A few lectures 

d. ) Never 
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STATEMENT _ | Strongly Agree _ | Somewhat Agree | Somewhat Disagree | Strongly Disagree 

4. ) Using a laptop increased my attentiveness in this class relative to other classes 

5. ) Using a laptop increased my engagement in this class relative to other classes 

6. ) My learning has been positively affected because of using a laptop in this class 

7. ) I spend more time on tasks unrelated to class/lecture than in other classes because of using a laptop in this 

_ class _ 

8. ) I have asked at least one question, via Lecture Tools, during class this semester. 

Yes 

No 

9. ) How distracted were you by other students’ laptops? 

a. ) Not distracted 

b. ) Somewhat distracted 

c. ) Distracted 

d. ) Very distracted 

10. ) What else, if anything distracted you during class? 

11. ) On a scale of 1-5, where 1 is very low and 5 is very high, to what degree was any lack of attentiveness in 

this class due to technical difficulties with your laptop? 

a. ) 1- Very low impact on attentiveness 

b. ) 2-Low impact on attentiveness 

c. ) 3- Medium impact on attentiveness 

d. ) 4-High impact on attentiveness 

e. ) 5-Very high impact on attentiveness 

12. ) On a scale of 1-5, where 1 is very low and 5 is very high, to what degree was any lack of attentiveness in 

this class due to technical difficulties with the instructors’ presentation? 

a. ) 1- Very low impact on attentiveness 

b. ) 2-Low impact on attentiveness 

c. ) 3- Medium impact on attentiveness 

d. ) 4-High impact on attentiveness 

e. ) 5-Very high impact on attentiveness 


66 


© 2012 The Clute Institute 




